1. Field of the Invention
The present invention relates to circuitry, an apparatus and a method for the conversion of a decoded image and also to a data recording medium capable of being read by a computer and storing a decoded image conversion program. The present invention is applicable to circuitry and an apparatus for decoding data coded by an MPEG (Moving Picture Coding Experts Group) 2 or similar standardized system and displaying an image represented by the decoded data.
2. Description of the Background Art
It is a common practice with an image coding system based on the MPEG2 system to divide one frame of picture into macroblocks each having 16xc3x9716 pixels. Redundant information included in the frame are compressed in the time domain macroblock by macroblock. In addition, to implement a random access function and a high coding efficiency, the MPEG2 type system defines three different picture types, i.e., an I (Intra coded) picture, a P (Predictive coded) picture, and a B (Bidirectionally predictive coded) picture on a frame basis.
An I picture is coded within a frame (intra coding) independently of the other pictures. All the macroblocks constituting an I picture are subjected to intra coding. By arranging such I pictures periodically, it is possible to effect random access or to use I pictures as error recovery pointers. While frequent appearance of I pictures lowers the total coding efficiency of the coding system, a higher coding efficiency is achievable with I pictures at scene switchover points and for images with a low prediction efficiency. A P picture is a forward predictive coded picture, i.e., subjected to predictive coding using an I picture or a P picture appeared in the past. A B picture is a bidirectionally predictive coded picture, i.e., subjected to predictive coding using I pictures or P pictures included in one or both of two pictures preceding and following the B picture; the direction of prediction is selected macroblock by macroblock. Even in a region where a certain object appears or disappears, B pictures allow predictive coding to be accurately effected by use of preceding and/or following pictures and noticeably enhance the coding efficiency.
I, P and B pictures may be combined in any desired format feasible for the object of a coding apparatus. As for the coding order, I pictures and P pictures following B pictures with respect to time are coded before the B pictures. FIG. 2 shows a specific original image represented by a sequence of pictures B0, B1, 12, B3, B4, P5, B6, B7, P8, B9, B10, P11, B12, B13 and B14. These pictures are coded in the order of 12, B0, B1, P5, B3, B4, P8, B6, B7, P11, B9, B10, P14, B12 and B13, as shown below the above original sequence in FIG. 2. The coded picture sequence is written to a recording medium and then decoded in the same order as it has been coded, as shown in the middle part of FIG. 2. Finally, the decoded sequence is rearranged in the original order, i.e., B0, B1, 12, B3, B4, P5, B6, B7, P8, B9, B10, P11, B12, B13 and B14 and then displayed as a reproduced image.
FIG. 3 shows conventional decoding circuitry based on the MPEG2 system. As shown, a coded bit stream coming in through an input port 40 is written to a receipt buffer 41. A VLC (Variable Length Code) decoder 42 separates the coded data into various kinds of data. Among the separated data, quantized DCT (Discrete Cosine Transform) coefficients are dequantized by a dequantizer 43 to turn out DCT coefficients. The DCT coefficients are subjected to inverse DCT transform by an inverse DCT 44. If the image data are representative of an I picture, then they are directly written to one of two frame memories (FM1 and FM2) 45 and 46.
Assume that the image data output from the inverse DCT 44 are representative of a P picture. Then, image data subjected to motion compensation by a motion compensation circuit (MC1) 47 in accordance with motion vectors are read out of the frame memory 45. Of course, such data may be those subjected to motion compensation by the other motion compensation circuit (MC2) 48 and read out of the other frame memory 46. The image data read out of the frame memory 45 are selected by a switch 34 and added, pixel by pixel, to the image data output from the inverse DCT 44 by an adder 50. The image data output from the adder 50 are written to the other frame memory 46. When the image data output from the inverse DCT 44 are representative of a B picture, image data respectively subjected to motion compensation by the motion compensation circuits 47 and 48 in accordance with motion vectors are respectively read out of the frame memories 45 and 46. In this. case, an averaging circuit (A) 49 averages the image data read out of the frame memories 45 and 46 and outputs the resulting mean values in the form of pixel values. The pixel values are selected by the switch 34 and added, pixel by pixel, to the image data output from the inverse DCT 44 by the adder 50. However, the reproduced image derived from the B picture is written neither to the fame memory 45 nor to the frame memory 46. The frame memories 45 and 46 are assumed to be used alternately.
The B picture reproduced by the above processing is directly output via an output port 36. The I and P pictures also reproduced by the above processing each is read out of the associated frame memory 45 or 46 and then output via the output port 36. As a result, a reproduced image is successfully displayed in the same order as the original image. A switch 35 selects one of the outputs of the adder 50, and frame memories 45 and 46 at a time.
In practice, however, the decoding circuitry based on the MPEG2 system is usually constructed to output display image data in the interlacing order. Therefore, the decoding circuit of the type outputting image data in the form of macroblocks must be followed by a memory for macroblock/raster scan conversion. Moreover, if the picture coding type has a frame structure, as distinguished from a field structure, then the above memory must be capable of storing at least half a field of image data when image data are output by interlacing.
To solve the above problem, Japanese patent laid-open publication No. 59084/1995 discloses an image processing system capable of dealing with compressed image data fed thereto in the form of packets. Specifically, as shown in FIG. 4, the image processing system taught in the above document includes three frame memory areas FM1, FM2 and FM3 exclusively assigned to I pictures, P pictures and B pictures, respectively. With these frame memory areas FM1, FM2 and FM3, it is possible to output image data in the interlacing order even when the picture coding type has a frame structure. More specifically, the image processing system uses four picture pointers RP, FP, BP and DP (see FIGS. 5 and 6) in order to determine the frame memory area FM1, FM2 or FM3 which a memory controller MCU should access. The picture pointers RP, FP, BP and DP respectively point at the frame memory areas where a current reproduced picture, a forward picture, a backward picture, and a current display picture are stored.
FIG. 5 demonstrates the transition of the picture pointers RP, FP, BP and DP occurring when pictures of different types are decoded and displayed. At the time when the first picture I0 is decoded, the image is not displayed yet. At this instant, the reproduced picture pointer RP points at an unoccupied area for storing the picture I0, e.g., the frame memory area FM1.
When a picture P1 following the picture I0 is decoded, the picture I0 is displayed without fail. At this instant, the reproduced picture pointer RP points at, e.g., the frame memory area FM2 while the display picture pointer DP points at the frame memory area FM1 storing the picture I0. Because a predictive picture for P1 need the forward picture I0 at the time of reproduction, the forward picture pointer FP also points at the fame memory area FM1.
When a picture B2 is decoded, it is also displayed. Both the reproduced picture pointer RP and display picture pointer DP point at the unoccupied frame memory area FM3. For the picture B2, the forward image I0 and backward image P1 are necessary at the time of decoding. At this instant, the forward picture pointer FP and backward picture pointer BP point at the frame memory areas FM1 and FM2, respectively.
It is a common practice with a display to delay an image by substantially half an image, so that the image can be displayed when decoded. Even when image data are read out every other line, i.e., by interlacing at the time of display despite that the picture coding type has a frame structure, the above delay of the image allows the decoding of image data located at a position to be read out for display to complete before display without fail.
A picture B3 is also displayed when decoded. Because the picture B3 needs the pictures I0 and P1 at the time of decoding, the forward picture pointer FP and backward picture pointer BP point at the frame memory areas FM1 and FM2, respectively. The reproduced picture pointer RP points at the frame memory area FM3, so that the picture B3 is written to the area FM3. The display picture pointer DP points at the frame memory area FM3 as it did for the picture B2.
The picture B2 stored in the frame memory area FM3 is displayed at the same time as the decoded picture B3 is written to the area FM3. In the case where the reproduced picture B3 is about to be written over the picture B2 to be displayed, a variable length decoder (VLD) 51 (see FIG. 4) delivering the data of the picture B3 to the frame memory area FM3 is deactivated by a sequencer (SEQ) 63 (see FIG. 4). The sequencer 63 controls the variable length decoder 51 with an enable signal such that the decoded macroblock position of the picture B3 does not exceed the display position of the display picture B2.
When a picture P4 is decoded, the picture P1 is displayed without fail. The picture P4 is written to the frame memory area FM1 which become idle later. At this instant, the display picture pointer DP points at the frame memory area FM2 to store the picture P1. The picture P4 needs the forward picture P1 at the time of decoding. The forward picture pointer FP points at the frame memory area FM2.
When a picture B5 is decoded, it is also displayed without fail. The picture B5 is written to the frame memory area FM3 which becomes idle. At this instant, the reproduced picture pointer RP and display picture pointer DP both point at the frame memory area FM3. The picture B5 needs the forward picture P1 and the backward picture P4 having already been decoded. The forward picture pointer FP and backward picture pointer BP point at the frame memory areas FM2 and FM1, respectively.
However, the conventional image processing system described above has the following problem left unsolved. As shown in FIG. 6, to decode a B picture, the system must read out two frames of images (forward predictive image and backward predictive image) at maximum as reference images within a single frame period, then store one frame of decoded image, and then read out one frame of image to be displayed. The system therefore needs a frame memory bus capable of transferring four frames of images within a single frame period for the decoding of an image and the conversion (display) of a decoded image. A frame memory bus with such a capacity corresponds to a frame memory bus MBUS shown in FIG. 4. Consequently, it is necessary to increase the bus width (bit width) of the frame memory for implementing rapid access to the frame memory or to use a frame memory capable of being rapidly accessed, resulting in an increase in the cost of the system.
It is therefore an object of the present invention to provide circuitry, an apparatus and a method for the conversion of a decoded image and capable of converting decoded image data to a sequence of image data for display (raster scan data) with a simple construction by reducing access to a frame memory and without resorting to a greater bus width.
It is another object of the present invention to provide a data recording medium capable of being read by a computer and storing a decoded image conversion program.
In accordance with the present invention, decoded image converting circuitry converts, on receiving decoded image data produced by dividing a frame image representative of a picture rearranged in accordance with the picture type into a plurality of small areas of image data, coding the small areas of image data, and decoding the resulting coded image data, the decoded image data to raster scan data area by area. The circuitry includes a compressing circuit for dividing the decoded image data of each small area into pixel data sequences each extending in the horizontal direction in a frame image, and compressing each pixel data sequence to thereby output compressed image data. A first storage stores, among the compressed image data output from the compressing circuit, compressed image data representative of an I picture and compressed image data representative of a P picture. A second storage stores, among the compressed image data output from the compressing circuit, compressed image data representative of a B picture. An expanding circuit takes in the compressed image data stored in the first and second storages in the order of the original picture and then expands them to thereby output the raster scan data.
Also, in accordance with the present invention, a decoded image converting apparatus includes an inverse transforming circuit for executing, on receiving coded image data coded by rearranging a picture in accordance with the picture type and subjecting the resulting picture to quantization and orthogonal transform, dequantization and inverse orthogonal transform with the coded image data to thereby output image data. A memory includes two memory areas for storing decoded image data respectively representative of an I picture and a P picture. An adder outputs, on receiving image data representative of an I picture from the inverse transforming circuit, the image data as decoded image data and writes them in one of the two memory areas of the memory. On receiving image data representative of a P picture from the inverse transforming circuit, the adder reads image data representative of an I picture or a P picture preceding the above P picture out of one of the two memory areas, adds the image data read out to the image data output from the inverse transforming circuit, and outputs the resulting decoded image data while writing them in the other memory area of the memory. Further, on receiving image data representative of a B picture from the inverse transforming circuit, the adder reads image data out of the two memory areas, produces mean values, and adds the mean values to the image data output from the inverse transforming circuit to thereby output decoded image data. A compressing circuit divides the decoded image data output from the adder into pixel data sequences each extending in the horizontal direction in a frame image, and compresses each pixel data sequence to thereby output compressed image data. A first storage stores, among the compressed image data output from the compressing circuit, compressed image data representative of an I picture and compressed image data representative of a P picture. A second storage stores, among the compressed image data output from the compressing circuit, compressed image data representative of a B picture. An expanding circuit takes in the compressed image data stored in the first and second storages in the order of the original picture and then expands them to thereby output raster scan data.
Further, in accordance with the present invention, a decoded image converting method converts decoded image data produced by dividing a frame image representative of a picture rearranged in accordance with the picture type into a plurality of small areas of image data, coding the small areas of image data, and decoding the resulting coded image data to raster scan data area by area. The method begins with the step of dividing the decoded image data of each small area into pixel data sequences each extending in the horizontal direction in a frame image, and compressing each pixel data sequence to thereby output compressed image data. Among the compressed image data output in the above step, compressed image data representative of an I picture and compressed image data representative of a P picture are written to a first storage while compressed image data representative of a B picture are written to a second storage. The compressed image data stored in the first and second storages are read out in the order of the original picture and then expanded to thereby output the raster scan data.